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ABSTRACT 

This paper reviews two areas of limitations 
concerning college outcome assessments and argues that while 
cognitive outcomes may be effectively measured during the students' 
academic careers, the perceptions of alumni are a necessary component 
of the assessment of affective outcomes . The limitations that are 
addressed concern first, determining what outcomes are to be 
assessed, and second, deciding who to assess (current students or 
alumni) . Both the theoretical aspects of college outcomes assessment 
and the application of the assessment method are discussed. The 
theoretical section addresses the nature of cognitive and affective 
objectives and the appropriate role of students and alumni m the 
assessment of these outcomes. The applied section makes reference to 
the assessment method assumed by an outcomes study undertaken during 
the 1980s by 13 church-related colleges in determining the 
achievement of affective outcomes as measured by the perceptions of 
over 3,000 alumni. Attention is focused on the method of assessment 
rather than on an analysis of study results. Contains 15 references. 
(GLR) 
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THE ROLE OF ALUMNI IN THE ASSESSMENT 
OF AFFECTIVE OUTCOMES 

by 

David Lee Parkyn 
Associate Professor of Religion 
Director of General Education 
Messiah College 
Grantham, Pennsylvania 

The values and attitudes which will characterize American society in the future 
depend, in part, on the kind of people the students in today's institutions of higher 
learning become. Consequently, it is important for colleges to frequently ascertain the 
extent to which they are effecting the various developmental objectives associated with 
the learning and teaching context. This is an endeavor which goes to the heart of the 
educational enterprise: As educators do we accomplish the outcomes for which higher 
education is responsible? 

While outcomes assessment has been part of higher education for numerous 
decades, the recent increased attention to the study of educational outcomes is due 
primarily to the new vigor with which the academy's various constituencies are holding 
it accountable. As Peter Ewell has noted, "across the country, powerful voices have 
begun clamoring for institutional accountability in higher education, accountability based 
on measuring 'objective' outcomes. " ; These voices have arisen as a result of both a 
concern within colleges and universities themselves as well as from pressure placed on 
them by the public at-large. The concern calls upon the educational institution to 
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prove itself as a valuable and worthwhile enterprise. 

All assessment efforts are, by their very nature, limited in particular ways. The 
researcher must minimally determine what outcomes are to be assessed and on what 
specific constituency of higher education these outcomes will be measured. My 
intention in this paper is to review these two areas of limitation, and through doing so, 
to argue that while cognitive outcomes may be effectively measured during the students' 
academic careers, the perceptions of alumni are a necessary component of the 
assessment of affective outcomes. 

This thesis will be developed in both theoretical and applied sections of the 
paper. In the theoretical component, the issues addressed include the nature of 
cognitive and affective objectives and the appropriate role of students and alumni in 
the assessment of these outcomes. In the applied section, reference is made to the 
assessment method assumed by an outcomes study undertaken during the 1980s by 
thirteen church-related colleges in determining the achievement of affective outcomes 
as measured by the perceptions of over 3000 alumni. Attention in this section is 
focused on the method of assessment rather than on an analysis of the findings of the 
study. 

WHAT DO WE ASSESS? 

A cursory reading of a random selection of statements of philosophy printed 
near the beginning of most college catalogs will indicate that as institutions of higher 
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learning we expect to accomplish a variety of student outcomes. Not only do we want 
our students to have depth of knowledge in at least one field, we also require that they 
be broadly informed in a variety of disciplines. Not only do we expect to teach our 
students to be concerned with physical fitness and recreative use of their leisure time, 
we also desire to advance their aesthetic responsiveness and discernment. Not only do 
we endeavor to assist our students in obtaining the prerequisites for vocational 
competence, we also seek to cultivate their avocational interests and skills. Not only 
do we posit the goal of developing in our students a deep sense of ethical and social 
concern, we also assume the responsibility for facilitating their adjustment to the 
stresses inherent in life. Not only do we wish to help our students recognize that 
learning is a life-long process and equip them to be able to pursue this process 
independently, we also hope to lay the foundation for their ongoing physical, spiritual, 
social, and emotional health. 

Colleges and universities expect to affect students in a wide variety of ways. 
What we expect to accomplish in the life of our students is a series of diverse and, at 
times, self-contradictory expectations. Given the nature of our mission we must 
repeatedly ask whether the successful fulfillment of these glowing expectations is 
documented in the lives of our students and alumni. 

Any single assessment effort cannot assume the responsibility to examine all of 
these outcomes. It is best to group them into selected categories or typologies and to 
assess them accordingly, through methods and instruments designed specifically for 
these purposes. 
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One of the most familiar ways of doing this is the taxonomy in which outcomes 
are categorized as either cognitive or affective. 2 Alexander Astin explains that 
"cognitive measures have to do with behavior that requires the use of high order 
mental processes such as reasoning and logic; affective measures deal with behavior 
that relates to the student's attitudes, feelings, and personality." 3 Cognitive outcomes, 
thus, concern the acquisition of formal knowledge, the level of academic achievement, 
the refinement of critical thinking skills, and the development of vocational skills. 
Affective outcomes include the development of self-concept, values, and beliefs, as well 
as a concern for avocations, satisfactory mental health, participatory citizenship, and 
fulfilled interpersonal relationships. 

Another frequently cited classification of outcomes dimensions is that suggested 
by Peter Ewell.^ He maintains that assessment should concentrate on three areas- 
knowledge, skills, and attitudes/values-followed by a fourth category dealing with the 
students' relationships with the larger society. The first two of these are similar to 
Astin's cognitive designation, while Ewell's third group is similar to what Astin describes 
as affective outcomes. 

Cognitive outcomes, or those related to knowledge and skills, are most often 
associated with the curriculum and the classroom as the explicit context for teaching 
and learning. These objectives serve as a significant part of the rationale for general 
education and also influence the manner in which certain parts of major programs are 
designed. Thus cognitive outcomes are often associated with speaking and writing in a 
clear, correct, and effective manner; with critical and quantitative thinking and 
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reasoning skills; with intellectual growth in a variety of academic disciplines; and with 
preparation for employment in a specific vocational or professional area. 

By contrast, affective outcomes, or those most directed at attitudes and values, 
are generally associated with a cross-section of campus experiences. In this sense they 
are more difficult to define, categorize, and measure. They include aspects of 
individual growth in the areas of aesthetics, morality, social adaptability, personal 
development, and spirituality. 

Over a decade ago, in his landmark study Investment in Learning . Howard 
Bowen maintained that the fulfillment of both cognitive and affective outcomes has a 
positive effect on both the individual students and society-at-large. His research 
suggested that "college education significantly raises the level of knowledge, the 
intellectual disposition, and the cognitive power of its students." 5 Furthermore, in the 
affective realm, Bowen concluded that college education helps students to develop 
personal identity, psychological well-being, increased tolerance toward other ethnic 
groups and "practical competence ... as citizens, workers, family members, and 
consumers." 6 

Since the various types of outcomes differ significantly it is usually preferable for 
the researcher to concentrate on a single type, rather than studying all of them 
together. Part of this is due to the distinctive features of the various outcomes, but 
much of it relates to the question concerning which of the constituencies of higher 
education is best able to assess the level of success we have had in attaining each of 
the proposed outcomes. 
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WHOM DO WE ASSESS? 

Outcomes assessment is generally accomplished through information gathered 
from either currently enrolled students or alumni. The majority of assessment activities 
are based on measurements undertaken during the students' college years. The goal of 
student-based studies is to investigate whether or not "students, regardless of who they 
are or where they go to college, typically change in definable ways during their 
undergraduate years." 7 When currently enrolled students are utilized to measure 
outcomes, the most popular method of measurement makes use of a freshman pre-test 
and senior post-test instrument. This methodology is appropriate for studies measuring 
cognitive outcomes and can also be used to measure certain affective outcomes. 

Among the significant shortcomings of this methodology, however, particularly in 
its adaptation to affective studies, concerns the failure to distinguish between college- 
based developmental influences and the developmental impetus students receive over 
the course of their years in college which comes from sources other than the college 
experience per se. In addition, the pre-test/post-test method does not allow for those 
changes which are based on a normal maturation process, those which would have 
occurred regardless of whether or not the individual studied at the undergraduate 
level* 

The use of alumni as a data base for outcomes studies is less popular than 
assessment via currently enrolled students. Of the many outcomes studies reviewed by 
Feldman and Newcomb 9 research with alumni was limited to the degree of persistence 
of college-incurred changes. No study was reviewed which was designed to discover 



precisely what these changes were, nor, more specifically, to what extent the college 
experience had influenced said changes. Robert Pace 70 has suggested that historically 
there have been occasional alumni-based studies, but asserts that these have generally 
tended to be poorly organized and limited in their orientation. 

Alumni-based outcome* assessment is important lor at least four reasons. First 
it provides the opportunity .'or "detached objectivity." When students are the source of 
assessment their perspective is often limited due to the immediacy of the college 
environment. When alumni participate in assessment activities their measurement of 
outcomes is often more reflective, given their distance from the college setting, both 
geographically and temporally. 

Second, alumni-based outcomes assessment is useful because it can effectively 
measure the perceptions of the participants. Perceptions are an important component 
in affective outcomes assessment. This type of research does not measure concrete 
actions, as is the case when the number of active registered voters are understood to 
be a measure of responsible citizenship. Rather, alumni-based research can ask for the 
graduate's perception concerning the extent to which the undergraduate experience 
cultivated responsible citizenship, however it may be objectively expressed in the 
individual's life. Both Robert Pace and Peter Eweli assert that there is a positive 
correlation between a graduate's perception and actual levels of achievement. 7 1 

Third, this type of assessment permits alumni to differentiate between 
developmental outcomes which were the result of the undergraduate experience and 
environment and those which were promoted principally through contributing forces 
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beyond the educational setting. While the graduates' ability to distinguish between 
these two is not nearly absolute, the measurement of their perceptions in this regard is, 
nonetheless, a valuable source of outcomes data for the college l» university. 

Finally, while those assessment endeavors which measure currently enrolled 
students focus on short-term outcomes alumni-based research provides an appropriate 
context within which to measure long-term objectives. Assessment on this level is 
important as many of higher education's objectives apply not only to the student but to 
the individual throughout life. And, as Peter Ewell has noted, "assessment and 
communication of long-term objectives . . . probably has the greatest potential for 
influencing public perception of the value of higher education as a whole. ,,/2 

If what we have asserted is correct then we must ask why assessment efforts do 
not more frequently include alumni. Certainly, currently enrolled students are more 
accessible to researchers, but convenience of accessibility should not overly restrict 
significant outcomes assessment. Periodic assessment efforts should ask how alumni 
who are five, ten, fifteen, or twenty years removed from their undergraduate studies 
perceive the influence of their college experience on their life. As Robert Pace has 
suggested, "I should think one would want to know how college graduates evaluate 
their own educational experience and how they view higher education as a social 
institution."^ Alumni provide the bottom line for the academy. How satisfied are 
alumni with their undergraduate experience once they have had some reasonable time 
to live with the results of their education and reflect upon it? 
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Cognitive outcomes, particularly those which concern intellectual development, 
are well measured throughout the students' academic careers. The persistence of 
cognitive outcomes and the fulfillment of vocationally oriented outcomes are best 
measured through alumni-based studies. When affective outcomes are to be measured 
the matured and detached perspective of alumni is to be preferred over the perspective 
of current students. 

Alumni-based studies traditionally have been directed primarily to the 
measurement of vocationally oriented cognitive outcomes. In recent years, however, it 
has been acknowledged that we should not limit alumni-based studies to the cognitive 
realm. As a result, affective outcomes have been the focus of a growing number of 
studies. This is particularly the case for those colleges which emphasize the liberal 
arts; researchers at these schools operate under the assumption that "many objectives 
of liberal education are phrased in a way that assume an integration within the 
individual of knowledge, value, and behavior,"^ and this integration Is best perceived 
when the students' perspective on their educational experiences has been allowed to 
mature. 

AN APPLIED MODEL OF ALUMNI -BASED ASSESSMENT 

Outcomes assessment for church-related liberal arts colleges presents a particular 
challenge. Students choose to attend these schools not so much for the cognitive 
development which can take place there, and which they fully expect to take place; 
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they enroll, rather, because of the possibilities for affective development which are here 
available. Students can learn the complexities of computer science, physics, literature, 
art, history, psychology, etc. on any college campus. But not all colleges have the 
environment for nor claim to produce the many affective outcomes that church-related 
colleges assume. 

A review of the statements of philosophy which church-related colleges print in 
their catalogs confirms that these colleges maintain, in theory at least, a significant 
emphasis on the affective development of their students. If this is the case, then these 
colleges bear the responsibility for excellence in the affective components of 
development as well as the cognitive. 

Since 1985 I have collaborated in an assessment project with a group of thirteen 
private liberal arts colleges, each with an enrollment of less than 2500 students, and 
each with an historic and present commitment to education within a church-related 
context. Though geographically distributed throughout the United States, these are 
generally regional colleges, with most students on each campus coming from states 
contiguous to that in which the college is located. The colleges have undertaken 
various collaborative efforts in recent years through a consortium they established in 
the late 1960s. 

While the thirteen colleges in this study affirm their commitment to the affective 
formation of their students, very little formal research has been undertaken on these 
campuses to determine the extent to which they actually cultivate for their students the 
affective domain. Prior to the present study these colleges generally relied on 
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anecdotal tradition to explain to their constituencies, and to themselves, the success 
which they achieved in the affective formation of students. 

For the present study, an assessment instrument was designed based principally 
on the statements of philosophy presented by the colleges in their respective academic 
catalogs. While many of these colleges have developed other documents which specify 
more extensively the nature of the expected outcomes, these statements of philosophy 
served as the common ground between the colleges. Had the instrument been 
designed for a single institution, the associated documents pertinent to a specific college 
may well have provided a more extensive basis upon which to develop an assessment 
instrument. 

The assessment instrument for this study contained a series of twenty-eight 
statements, each set forth in response to a single question, "In thinking back to your 
undergraduate education at this college, to what extent do you think you were 
influenced in each of the following areas?" For each statement the respondents 
selected one of four levels of perceived influence, ranging from none to very much. 
While no systematic attempt is made in the individual college statements of philosophy 
to define clearly developed affective constructs, each of the twenty-eight statements, as 
developed for the assessment instrument, was designated as a component of one of five 
affective developmental areas. These five areas included statements relating to 
aesthtuc development, moral development, personal development, social development, 
and spiritual development. 
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The instrument was designed to be administered by mail. This method was 
preferred over other options (e.g., telephone interviews and in-person interviews) as it 
provides an acceptable response rate with high quality respOii*~» at a moderate cost. 
Alumni from these colleges are presently living in all fifty states and several foreign 
countries. As such, in-person interviews would have been very costly and time 
consuming. Telephone interviews could have been conducted, but while these would 
have assured a higher response rate, there is a higher potential for bias in the 
responses received.^ 

The alumni included in this assessment project were those who were members of 
the five year anniversary class. This class was selected for two reasons. First, it was 
assumed that the perceptions of graduates would most accurately reflect the influence 
of the college when they had lived away from the college environment for some 
extended period of time, providing them with an opportunity to gain a moderately 
detached perspective on their undergraduate experience. Second, while it may be 
argued that an even greater time-span is needed to provide for a more matured 
perspective on one's undergraduate experience (hence a ten or fifteen year anniversary 
class should be surveyed), this would have proven, in the end, to be a detriment to this 
study as most of the participating schools have changed significantly over the recent ten 
to fifteen years. 

Three years following the first administration of this assessment project, a 
subsequent distribution of the instrument was made to the graduating class which was 
then celebrating its fifth anniversary. In these two administrations of the project, over 
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6000 alumni have been contacted, with nearly 3300 respondents, for a response rate of 
over 55 percent. Plans are in place for a third administration of the study, providing a 
longitudinal context spanning three graduating classes, each separated by three years. 

Response data were analyzed in a variety of ways. Statements were associated 
in developmental areas, and response means were analyzed accordingly. Various 
comparative groups were also used, including male and female alumni, alumni who had 
attended the college for four years and those who had transferred from another 
institution, and alumni who had resided in on-campus housing and those who had 
commuted to the campus. These analyses were accomplished for data from each 
college and for all respondents collectively. In addition, following the second 
administration of the study, the data were analyzed on a longitudinal basis to determine 
the manner in and extent to which the perceptions of alumni, both collectively and for 
individual colleges, differed between the two classes. 

An alumni assessment effort of this nature could be effectively transferred to any 
college. If this were the case, several modifications to the procedure should be 
considered. Among these are the following. 

First, the assessment instrument could be designed in a way which more 
accurately reflects the particularities of the individual campus. Statements reflecting the 
intentional outcomes of the particular school would be an essential component of the 
instrument. The faculty to which I belong, for example, has recently adopted a 
statement of College-Wide Educational Objectives which are to be the guiding 
principles upon which both curricular and co-curricular programming will be initiated 
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and reviewed in the forthcoming years. The objectives delineated in this statement 
include both cognitive and affective outcomes, as this is the educational tradition upon 
which this college was established. The specific objectives included in this statement 
should serve as the basis for any future assessment effort. Second, a means other 
than a mailed instrument might be appropriated. Telephone contact with a random 
sample of alumni in a given graduating class could obtain a high response rate. Such 
an effort could be accomplished at a moderate cost, particularly if the college has a 
large commuter population, the graduates of which may have generally remained in the 
local area. The gathering of responses by telephone would, of course, be dependent 
on the availability of appropriate telephone numbers, and special care would need to 
be taken to guard against the potential for bias which exists in this context. 

Finally, the comprehensive nature of the assessment effort would be diminished, 
as there would be no basis of comparison with other like schools. The lack of 
comparative data could be modified, however, if the instrument administered were 
associated with an established assessment project, such as the NCHEMS/College Board 
Student Outcomes Information Service or the ACT Evaluation Survey Service. Both of 
these are designed, in part, to measure affective outcomes, have instruments available 
for administration to alumni, and provide for the addition of local questions to the 
standardized questionnaire. This would provide a basis for some comparison, although 
it would still be limited to those items on the national instrument, and would not 
achieve a comparison on local additions. 

in 
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A CONCLUDING ASSERTION 

All colleges are, to varying degrees, concerned with the development of their 
students. The intellectual/cognitive development of the college student has been the 
basis for significant assessment in the past, and this type of study will and should be 
continued in the future. Yet the very nature of such assessment suggests that the 
cognitive domain is the most, if not the only, significant aspect of the development of 
students. To study the cognitive development of students, however, while ignoring their 
affective development fails to assess a major component in the objectives of higher 
education. Likewise, to study the affective domain and ignore the cognitive is equally 
inadequate. We must not concentrate on one and ignore the other. 

As educators, we should measure outcomes through both current students and 
alumni. I have maintained herein that current students should continue to be the basis 
upon which we measure cognitive outcomes. Historically, the study of alumni has been 
limited to either assessing the level of persistence of selected cognitive outcomes or to 
measuring a small number of vocationally oriented affective outcomes. The thesis of 
this paper asserts that affective outcomes should be increasingly measured by 
institutions of higher learning and that the perceptions of alumni are an important 
aspect of this assessment. 

Analysis of data received via alumni-based research can provide valuable 
information useful to the improvement of a college's program, both curricular and 
otherwise. At most colleges few attempts are undertaken to ask graduates how they 
perceive their undergraduate experiences to have influenced their lives. Surely colleges 
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have a responsibility to their graduates even after the last tuition payment has been 
made or the diploma has been presented. A continued effort at measuring the 
perceptions of alumni shows them that they are still a valued constituent for the college 
and permits current students to benefit from the insights gathered via such study. 
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